Significant damage, both reported and unreported, has been caused to enterprises, government agencies, and national security through “insider threat” attacks, especially data exfiltration. For example, consider recent retrieval and release of secret and top-secret information from the defense and intelligence communities by Manning and Snowden. Data exfiltration attacks may include, e.g., intentional user activity (e.g., a user impermissibly downloads sensitive data and removes the data from the enterprise) or automated activity (e.g., malicious software operates on behalf of or as a user with or without the user's knowledge). Unfortunately, the current state of the art for addressing such problems is quite limited.
1. Monitoring of all System Data: Due to the large volume of data that is required to monitor ALL users and systems in an enterprise, this data (often only portions of the required data) is usually stored and analyzed off-line in a database or data warehouse. Unfortunately, this analysis only reveals issues after the fact, i.e., after any actual data exfiltration attempts have occurred and at such a point in time when it may be too late to take any action or to prevent damage from the exfiltration. In a best case, the offending user or system is still there and may be expected to perform such actions again, where they can be targeted for further analysis and prosecution. However, many times, the offending user or system is no longer present, the damage has already been done or it is otherwise too late to take effective action.
2. Real-time Monitoring of Suspected Individuals: If an individual is suspected of malicious activity, then real-time monitoring mechanisms can be configured and installed to directly monitor that individual's activity and detect any malicious activity. These mechanisms, however, are time consuming to install and also require dedicated analysts to conduct the real-time monitoring and detection, often at great expense to the affected enterprise.
As noted above, there are significant problems with the above approaches. The above approaches are slow to react, may not catch a user in time, and have a hard time detecting malicious software that is installed and operating on behalf of a user without the user's knowledge or authorization. Furthermore, the above approaches to addressing data exfiltration problems are expensive to deploy and consume human analyst time and resources.
The insider threat remains one of the most significant problems confronting enterprises and government agencies of all sizes today. The threat is multi-faceted with a high degree of variability in the perpetrator, the type of attack, the intent of the attack, and the access means. No solution today adequately addresses the detection of insider threats due to the highly variable nature of the problem.
No existing systems or solutions takes user, database, application, and network activity all into account at the same time while using event processing techniques to discover patterns of behavior and anomalies from across these plurality of data streams in real-time in order to detect anomalies that could not have been detected by monitoring any single data stream alone.
Thus, it is desirable to provide a system and method for real-time detection of anomalies in database or application usage which are able to overcome the above disadvantages.